Skip to content

Try getting the error message in response_handler #640

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from

Conversation

alexandresobolevski
Copy link
Contributor

@alexandresobolevski alexandresobolevski commented Dec 21, 2016

[WIP, not ready for review...]

The issue tackled here is this https://github.com/plotly/streambed/issues/8801

I decided to give a hand and tackle it because it's part of 2.1 release that we want to deliver before Christmas and Adam is out on vacation.

Now the returned error is

PlotlyError: Sorry, a file already exists with this name.

@theengineear I'm failing test_duplicate_folders test but in an expected way I guess because I hit the above existing file error. I'd love suggestions about changing the test or modifying the PR for a sound result and a review is much appreciated since it's my first time in this repo.

@chriddyp fyi and 👀 ?

@chriddyp
Copy link
Member

Looks good to me!

@chriddyp
Copy link
Member

chriddyp commented Dec 22, 2016

I guess this is technically a backwards incompatible change so we should make a major version bump. Any code that is depending on error handling by subclassing requests.exceptions.RequestException will fail with this change as it is currently written.

Here is the test that is failing:

    def test_duplicate_folders(self):
        first_folder = self._random_filename()
        py.file_ops.mkdirs(first_folder)
        try:
            py.file_ops.mkdirs(first_folder)
        except requests.exceptions.RequestException as e:
            self.assertTrue(400 <= e.response.status_code < 500)
        else:
            self.fail('Expected this to fail!')

I like that the previous code's exception contained status_code. We can just modify the message or the __str__ of the requests.exceptions.RequestException but nothing else?

@theengineear
Copy link
Contributor

@chriddyp a couple thoughts. We have a PlotlyRequestException that is meant to take a requests.exceptions.RequestException and return the proper message (though I think it's outdated).

That means we have 3 ways to raise exceptions for requests:

  1. Use a generic PlotlyError
  2. Use a PlotlyRequestException
  3. Just raise the requests.exceptions.RequestException

I'd vote for not requiring a user to import requests and always raising PlotlyRequestException (and yes +1 to a version bump since this is backwards incompatible).

parsed_response = response.content

try:
# try get a friendly error message
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐐 try get

except:
raise requests_exception

raise exceptions.PlotlyError(message)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we just did this:

https://github.com/plotly/plotly.py/blob/master/plotly/plotly/plotly.py#L1704-L1720

Want to try and follow that more closely, OR, extract the common functionality out? It'd be great to just have one error handler.

In particular, the code you wrote assume that errors is not empty, which we might not want to assume.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this to the central error handler, I actually forgot we even had it!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to try and follow that more closely, OR, extract the common functionality out? It'd be great to just have one error handler.

that's what I was hoping to achieve, I took the main logic and moved it to the response_handler and use it in animations to catch errors instead of having a code block in animations for that.

In particular, the code you wrote assume that errors is not empty, which we might not want to assume

it's in a try bloc, so if my assumption is incorrect then it raises the usual requests_exception. I could put two if for looking to see if errors and message exist but would that not do the same?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aw jeez. i didn't realize it was still in a try block. that's fine then. But, ya that's what I was talking about. The lines I was linking to were:

    try:
        parsed_response = r.json()
    except:
        parsed_response = r.content

    # raise error message
    if not r.ok:
        message = ''
        if isinstance(parsed_response, dict):
            errors = parsed_response.get('errors')
            if errors and errors[-1].get('message'):
                message = errors[-1]['message']
        if message:
            raise exceptions.PlotlyError(message)
        else:
            # shucks, we're stuck with a generic error...
            r.raise_for_status()

@alexandresobolevski
Copy link
Contributor Author

I am still confused as to what I should do

I like that the previous code's exception contained status_code. We can just modify the message or the str of the requests.exceptions.RequestException but nothing else?

Doing this does not bump the version and I am not entirely sure how to change the message of requests.exceptions.HTTPError

I'd vote for not requiring a user to import requests and always raising PlotlyRequestException (and yes +1 to a version bump since this is backwards incompatible).

This bumps the version and I assume we're talking PlotlyRequestError, if so Im not sure how to get the status from the API in there.

this is what I have so far

                try:
                    # try get a friendly error message
                    errors = parsed_response.get('errors')
                    message = errors[-1]['message']
                except:
                    # otherwise raise a generic error
                    raise requests_exception
                else:
                    requests_exception.message = message
                    raise exceptions.PlotlyRequestError(requests_exception)

@theengineear
Copy link
Contributor

@alexandresobolevski , i was suggesting always raising the PlotlyRequestError (i believe that it sets the status code)

So, something more like:

                try:
                    # try get a friendly error message
                    errors = parsed_response.get('errors')
                    message = errors[-1]['message']
                except:
                    # otherwise raise a generic error
                    raise exceptions.PlotlyRequestError(requests_exception)
                else:
                    requests_exception.message = message
                    raise exceptions.PlotlyRequestError(requests_exception)

@theengineear
Copy link
Contributor

@chriddyp , I slacked with @alexandresobolevski a little bit on this. I'm going to hop in here unify how we do error handling. You're right that it requires a version bump to make this change, but I don't want to force users to change their error-handling code twice, that's silly.

alexandresobolevski and others added 3 commits December 22, 2016 13:27
In general, we have cyclic import issues all over the place, this is one
easy fix and will help out in later commits.

Note that this maintains backwards compat due to how the the functions
are imported into `plotly.py`.
This is also some setup for a larger refact. Since v1 and v2 requests
handle errors differently, it’s easier if we simplify the api into our
errors so that the requests can go from `response` —> `python error` as
they see fit.
@theengineear
Copy link
Contributor

@chriddyp ^^ here's a long-overdue Christmas miracle :p. I did a full overhaul on organizing our api calls in this repo. It's dead simple and unifies how both request creation and response validation work in both v1 and v2 code paths.

I also defer content parsing to the Response.json method in requests, which appears to by PY2/PY3 agnostic. This should greatly simplify problems with decoding/unicode schtuff. However, I'm going to let our test suite churn for now (may be a lot of brokenness).

And yep, this will be a big version bump. Again we were going to need a version bump as suggested in #640 (comment). I figured we may as well really tackle this error-handling thing if we're going to need to do a version bump anyhow.

@theengineear
Copy link
Contributor

Also, we're still only supporting 2.7.8 3.3.3 3.4.1, we should test 3.5 and 3.6 as well (Python devs are moving fast :p)

@theengineear
Copy link
Contributor

^^ 😮 holy $#!T! It passed already.

@theengineear
Copy link
Contributor

Manually test the following in Python 3:

  • loading graph reference
  • saving / retrieving plots with unicode

@theengineear
Copy link
Contributor

Just as a heads up, I tagged this as a WIP for now, I have a lot of tests that need adding in for the new api module I created. What I'd like to do is ensure that we have lots of tests with requests.request mocked and then just add a handful of tests that are actual integration tests. This test suite is unnecessarily slow right now, and will continue to feel slower as we test on more and more Python versions.

@theengineear theengineear force-pushed the handle-grid-errors branch 2 times, most recently from 3f893df to a436587 Compare December 28, 2016 20:16
@theengineear theengineear force-pushed the handle-grid-errors branch 3 times, most recently from c789a4f to 037eabb Compare December 29, 2016 19:12
This was driving me nuts. We basically manually handle creating and
validating *each* api response inside each calling function. Even worse,
we *sometimes* raise a `PlotlyRequestError` and *sometimes* just bubble
up the `requests.exceptions.HTTPError` ;__;.

This does the following:

* Define an `api.v1` module that only includes `clientresp` (the only old
api method we still *need* to cling to)
* Define an `api.v2` module that includes all the new functionality of
our v2 api.
* Both `v1` and `v2` raise the same `PlotlyRequestError`, so that users
only need to catch *one* exception class in scripts.
Note that the `apigetfile` did some weird things to convert old-style
plotlyjs figures (e.g. `type: ‘histogramx’`) to new-style versions.

I wanted to ween us off the old api, so this makes the change from
`/apigetfile` —> `/v2/plots/[fid]/content?inline_data=true`.

The `_swap*` functionality was copied from `plotly/streambed` code,
directly from the backend’s implementation of `apigetfile`.
There is a circular import issue that needed to be fixed here as well.
This is allowed when setting the credentials *file*, however, trying to
set this inside a session used to fail.
The `requests` package kindly manages 2/3 compat for json for us, might
as well be consistent and use the same tool!

As a side note, I’d like to move away from `six` and depend directly on
`requests.compat`. I believe it has everything we need and then we can
ditch the `six` dep and know that we’re always in sync with whatever
requests is doing (which is really what we care about).
@theengineear theengineear mentioned this pull request Dec 29, 2016
10 tasks
username = username.encode('latin1')

if isinstance(password, str):
password = password.encode('latin1')
Copy link
Member

@chriddyp chriddyp Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why latin1 and not utf-8?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd have to ask Kenneth Reitz. It's just something I pulled from requests.HTTPBasicAuth. My guess is that it doesn't matter, but I'd rather just leave it as was written there.

"""
url = build_url(RESOURCE, route='lookup')
params = make_params(path=path, parent=parent, user=user, exists=exists)
return request('get', url, params=params)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, this all looks really good

del entry['orientation']
figure['data'][index] = entry
except KeyError:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this to pass validation?

@@ -1738,7 +1554,7 @@ def icreate_animations(figure, filename=None, sharing='public', auto_open=False)
This function is based off `plotly.plotly.iplot`. See `plotly.plotly.
create_animations` Doc String for param descriptions.
"""
# Still needs doing: create a wrapper for iplot and icreate_animations
# TODO: create a wrapper for iplot and icreate_animations
Copy link
Member

@chriddyp chriddyp Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this comment applies anymore:

def icreate_animations(figure, filename=None, sharing='public', auto_open=False):

@chriddyp
Copy link
Member

chriddyp commented Jan 2, 2017

Wow, huge stuff! Looks great to me so far

@theengineear theengineear mentioned this pull request Jan 5, 2017
@theengineear
Copy link
Contributor

Closing in favor of #650

@theengineear theengineear deleted the handle-grid-errors branch January 5, 2017 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants